Papers by Alan Chi-Man Lee
PROMTEC: Fast LLM Inference Decoding using Prompt Multi-Lookup with Template Database and Common Sequences (2025.findings-acl)
Copied to clipboard
| Challenge: | Prompt Multi-Lookup, Template Datastore, and Common Sequences methods are used to accelerate the inference of large language models (LLMs). |
| Approach: | They propose a multi-faceted approach to accelerate the inference of large language models by leveraging Prompt Multi-Lookup, Template Datastore, and Common Sequences methods. |
| Outcome: | The proposed method achieves a 3.91 speedup on the miniF2F benchmark and 4.23 on the HumanEval benchmark. |